<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <title>Learning from Examples &raquo; Matrix Multiplication (cudaFlow) | Taskflow QuickStart</title>
  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600" />
  <link rel="stylesheet" href="m-dark+documentation.compiled.css" />
  <link rel="icon" href="favicon.ico" type="image/x-icon" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <meta name="theme-color" content="#22272e" />
</head>
<body>
<header><nav id="navigation">
  <div class="m-container">
    <div class="m-row">
      <span id="m-navbar-brand" class="m-col-t-8 m-col-m-none m-left-m">
        <a href="https://taskflow.github.io"><img src="taskflow_logo.png" alt="" />Taskflow</a> <span class="m-breadcrumb">|</span> <a href="index.html" class="m-thin">QuickStart</a>
      </span>
      <div class="m-col-t-4 m-hide-m m-text-right m-nopadr">
        <a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
          <path id="m-doc-search-icon-path" d="m6 0c-3.31 0-6 2.69-6 6 0 3.31 2.69 6 6 6 1.49 0 2.85-0.541 3.89-1.44-0.0164 0.338 0.147 0.759 0.5 1.15l3.22 3.79c0.552 0.614 1.45 0.665 2 0.115 0.55-0.55 0.499-1.45-0.115-2l-3.79-3.22c-0.392-0.353-0.812-0.515-1.15-0.5 0.895-1.05 1.44-2.41 1.44-3.89 0-3.31-2.69-6-6-6zm0 1.56a4.44 4.44 0 0 1 4.44 4.44 4.44 4.44 0 0 1-4.44 4.44 4.44 4.44 0 0 1-4.44-4.44 4.44 4.44 0 0 1 4.44-4.44z"/>
        </svg></a>
        <a id="m-navbar-show" href="#navigation" title="Show navigation"></a>
        <a id="m-navbar-hide" href="#" title="Hide navigation"></a>
      </div>
      <div id="m-navbar-collapse" class="m-col-t-12 m-show-m m-col-m-none m-right-m">
        <div class="m-row">
          <ol class="m-col-t-6 m-col-m-none">
            <li><a href="pages.html">Handbook</a></li>
            <li><a href="namespaces.html">Namespaces</a></li>
          </ol>
          <ol class="m-col-t-6 m-col-m-none" start="3">
            <li><a href="annotated.html">Classes</a></li>
            <li><a href="files.html">Files</a></li>
            <li class="m-show-m"><a href="#search" class="m-doc-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
              <use href="#m-doc-search-icon-path" />
            </svg></a></li>
          </ol>
        </div>
      </div>
    </div>
  </div>
</nav></header>
<main><article>
  <div class="m-container m-container-inflatable">
    <div class="m-row">
      <div class="m-col-l-10 m-push-l-1">
        <h1>
          <span class="m-breadcrumb"><a href="Examples.html">Learning from Examples</a> &raquo;</span>
          Matrix Multiplication (cudaFlow)
        </h1>
        <nav class="m-block m-default">
          <h3>Contents</h3>
          <ul>
            <li><a href="#GPUAcceleratedMatrixMultiplication">Define a Matrix Multiplication Kernel</a></li>
            <li><a href="#DefineAcudaFlowForMatrixMultiplication">Define a cudaFlow for Matrix Multiplication</a></li>
            <li><a href="#MatrixMultiplicationcudaFlowBenchmarking">Benchmarking</a></li>
          </ul>
        </nav>
<p>Following up on <a href="matrix_multiplication.html" class="m-doc">Matrix Multiplication</a>, this page studies how to accelerate a matrix multiplication workload on a GPU using <a href="classtf_1_1cudaFlow.html" class="m-doc">tf::<wbr />cudaFlow</a>.</p><section id="GPUAcceleratedMatrixMultiplication"><h2><a href="#GPUAcceleratedMatrixMultiplication">Define a Matrix Multiplication Kernel</a></h2><p>GPU can perform a lot of parallel computations more than CPUs. It is especially useful for data-intensive computing such as matrix multiplication. With GPU, we express the parallel patterns at a fine-grained level. The kernel, written in CUDA, is described as follows:</p><pre class="m-code"><span class="c1">// CUDA kernel to perform matrix multiplication</span>
<span class="n">__global__</span><span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="n">matmul</span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="o">*</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="o">*</span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="o">*</span><span class="n">C</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">M</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">K</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">N</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">  </span><span class="kt">int</span><span class="w"> </span><span class="n">row</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">blockIdx</span><span class="p">.</span><span class="n">y</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">blockDim</span><span class="p">.</span><span class="n">y</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">threadIdx</span><span class="p">.</span><span class="n">y</span><span class="p">;</span>
<span class="w">  </span><span class="kt">int</span><span class="w"> </span><span class="n">col</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">blockIdx</span><span class="p">.</span><span class="n">x</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">blockDim</span><span class="p">.</span><span class="n">x</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">threadIdx</span><span class="p">.</span><span class="n">x</span><span class="p">;</span>
<span class="w">  </span><span class="kt">int</span><span class="w"> </span><span class="n">sum</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w">  </span><span class="k">if</span><span class="p">(</span><span class="n">col</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">N</span><span class="w"> </span><span class="o">&amp;&amp;</span><span class="w"> </span><span class="n">row</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">M</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">    </span><span class="k">for</span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">K</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="o">++</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">      </span><span class="n">sum</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="n">a</span><span class="p">[</span><span class="n">row</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">K</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">i</span><span class="p">]</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">b</span><span class="p">[</span><span class="n">i</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">N</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">col</span><span class="p">];</span>
<span class="w">    </span><span class="p">}</span>
<span class="w">    </span><span class="n">c</span><span class="p">[</span><span class="n">row</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">N</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">col</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sum</span><span class="p">;</span>
<span class="w">  </span><span class="p">}</span>
<span class="p">}</span></pre><p>Each CUDA thread corresponds to an element of <code>C</code> and compute its result. Instead of storing each matrix in a 2D array, we use 1D layout to ease the data transfer between CPU and GPU. In a row-major layout, an element <code>(x, y)</code> in the 2D matrix can be addressed at <code>x * width + y</code> in the transformed 1D layout.</p><img class="m-image" src="matrix_multiplication_4.png" alt="Image" style="width: 70%;" /></section><section id="DefineAcudaFlowForMatrixMultiplication"><h2><a href="#DefineAcudaFlowForMatrixMultiplication">Define a cudaFlow for Matrix Multiplication</a></h2><p>The next step is to allocate memory for <code>A</code>, <code>B</code>, and <code>C</code> at a GPU. We create three tasks each calling <code>cudaMalloc</code> to allocate space for one matrix. Then, we create a cudaFlow to offload matrix multiplication to a GPU. The entire code is described as follows:</p><pre class="m-code"><span class="kt">void</span><span class="w"> </span><span class="nf">matrix_multiplication</span><span class="p">(</span><span class="kt">int</span><span class="o">*</span><span class="w"> </span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="o">*</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="o">*</span><span class="w"> </span><span class="n">C</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">M</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">K</span><span class="p">,</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">N</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w">  </span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Taskflow</span><span class="w"> </span><span class="n">taskflow</span><span class="p">;</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Executor</span><span class="w"> </span><span class="n">executor</span><span class="p">;</span>

<span class="w">  </span><span class="c1">// allocate the host and gpu storage for A</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Task</span><span class="w"> </span><span class="n">allocate_a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">taskflow</span><span class="p">.</span><span class="n">emplace</span><span class="p">([</span><span class="o">&amp;</span><span class="p">](){</span>
<span class="w">    </span><span class="n">cudaMalloc</span><span class="p">(</span><span class="o">&amp;</span><span class="n">da</span><span class="p">,</span><span class="w"> </span><span class="n">M</span><span class="o">*</span><span class="n">K</span><span class="o">*</span><span class="k">sizeof</span><span class="p">(</span><span class="kt">int</span><span class="p">));</span>
<span class="w">  </span><span class="p">}).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;allocate_a&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// allocate the host and gpu storage for B</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Task</span><span class="w"> </span><span class="n">allocate_b</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">taskflow</span><span class="p">.</span><span class="n">emplace</span><span class="p">([</span><span class="o">&amp;</span><span class="p">](){</span>
<span class="w">    </span><span class="n">cudaMalloc</span><span class="p">(</span><span class="o">&amp;</span><span class="n">db</span><span class="p">,</span><span class="w"> </span><span class="n">K</span><span class="o">*</span><span class="n">N</span><span class="o">*</span><span class="k">sizeof</span><span class="p">(</span><span class="kt">int</span><span class="p">));</span>
<span class="w">  </span><span class="p">}).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;allocate_b&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// allocate the host and gpu storage for C</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Task</span><span class="w"> </span><span class="n">allocate_c</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">taskflow</span><span class="p">.</span><span class="n">emplace</span><span class="p">([</span><span class="o">&amp;</span><span class="p">](){</span>
<span class="w">    </span><span class="n">cudaMalloc</span><span class="p">(</span><span class="o">&amp;</span><span class="n">dc</span><span class="p">,</span><span class="w"> </span><span class="n">M</span><span class="o">*</span><span class="n">N</span><span class="o">*</span><span class="k">sizeof</span><span class="p">(</span><span class="kt">int</span><span class="p">));</span>
<span class="w">  </span><span class="p">}).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;allocate_c&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// create a cudaFlow task to run the matrix multiplication</span>
<span class="w">  </span><span class="n">tf</span><span class="o">::</span><span class="n">Task</span><span class="w"> </span><span class="n">cudaFlow</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">taskflow</span><span class="p">.</span><span class="n">emplace</span><span class="p">([</span><span class="o">&amp;</span><span class="p">](){</span>

<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaFlow</span><span class="w"> </span><span class="n">cf</span><span class="p">;</span>
<span class="w">  </span>
<span class="w">    </span><span class="c1">// copy data to da, db, and dc</span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaTask</span><span class="w"> </span><span class="n">copy_da</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">cf</span><span class="p">.</span><span class="n">copy</span><span class="p">(</span><span class="n">da</span><span class="p">,</span><span class="w"> </span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">M</span><span class="o">*</span><span class="n">K</span><span class="p">).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;H2D_A&quot;</span><span class="p">);</span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaTask</span><span class="w"> </span><span class="n">copy_db</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">cf</span><span class="p">.</span><span class="n">copy</span><span class="p">(</span><span class="n">db</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">K</span><span class="o">*</span><span class="n">N</span><span class="p">).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;H2D_B&quot;</span><span class="p">);</span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaTask</span><span class="w"> </span><span class="n">copy_hc</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">cf</span><span class="p">.</span><span class="n">copy</span><span class="p">(</span><span class="n">C</span><span class="p">,</span><span class="w"> </span><span class="n">dc</span><span class="p">,</span><span class="w"> </span><span class="n">M</span><span class="o">*</span><span class="n">N</span><span class="p">).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;D2H_C&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">    </span><span class="n">dim3</span><span class="w"> </span><span class="n">grid</span><span class="w">  </span><span class="p">((</span><span class="n">K</span><span class="o">+</span><span class="mi">16-1</span><span class="p">)</span><span class="o">/</span><span class="mi">16</span><span class="p">,</span><span class="w"> </span><span class="p">(</span><span class="n">M</span><span class="o">+</span><span class="mi">16-1</span><span class="p">)</span><span class="o">/</span><span class="mi">16</span><span class="p">);</span>
<span class="w">    </span><span class="n">dim3</span><span class="w"> </span><span class="n">block</span><span class="w"> </span><span class="p">(</span><span class="mi">16</span><span class="p">,</span><span class="w"> </span><span class="mi">16</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaTask</span><span class="w"> </span><span class="n">kmatmul</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">cf</span><span class="p">.</span><span class="n">kernel</span><span class="p">(</span><span class="n">grid</span><span class="p">,</span><span class="w"> </span><span class="n">block</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">matmul</span><span class="p">,</span><span class="w"> </span><span class="n">da</span><span class="p">,</span><span class="w"> </span><span class="n">db</span><span class="p">,</span><span class="w"> </span><span class="n">dc</span><span class="p">,</span><span class="w"> </span><span class="n">M</span><span class="p">,</span><span class="w"> </span><span class="n">K</span><span class="p">,</span><span class="w"> </span><span class="n">N</span><span class="p">)</span>
<span class="w">                             </span><span class="p">.</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;matmul&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">    </span><span class="n">kmatmul</span><span class="p">.</span><span class="n">succeed</span><span class="p">(</span><span class="n">copy_da</span><span class="p">,</span><span class="w"> </span><span class="n">copy_db</span><span class="p">)</span>
<span class="w">           </span><span class="p">.</span><span class="n">precede</span><span class="p">(</span><span class="n">copy_hc</span><span class="p">);</span>

<span class="w">    </span><span class="c1">// launch the cudaFlow</span>
<span class="w">    </span><span class="n">tf</span><span class="o">::</span><span class="n">cudaStream</span><span class="w"> </span><span class="n">stream</span><span class="p">;</span>
<span class="w">    </span><span class="n">cf</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">stream</span><span class="p">);</span>
<span class="w">    </span><span class="n">stream</span><span class="p">.</span><span class="n">synchronize</span><span class="p">();</span>
<span class="w">  </span>
<span class="w">  </span><span class="p">}).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;cudaFlow&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// free the gpu storage</span>
<span class="w">  </span><span class="k">auto</span><span class="w"> </span><span class="n">free</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">taskflow</span><span class="p">.</span><span class="n">emplace</span><span class="p">([</span><span class="o">&amp;</span><span class="p">](){</span>
<span class="w">    </span><span class="n">cudaFree</span><span class="p">(</span><span class="n">da</span><span class="p">);</span>
<span class="w">    </span><span class="n">cudaFree</span><span class="p">(</span><span class="n">db</span><span class="p">);</span>
<span class="w">    </span><span class="n">cudaFree</span><span class="p">(</span><span class="n">dc</span><span class="p">);</span>
<span class="w">  </span><span class="p">}).</span><span class="n">name</span><span class="p">(</span><span class="s">&quot;free&quot;</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// create dependency</span>
<span class="w">  </span><span class="n">cudaFlow</span><span class="p">.</span><span class="n">succeed</span><span class="p">(</span><span class="n">allocate_a</span><span class="p">,</span><span class="w"> </span><span class="n">allocate_b</span><span class="p">,</span><span class="w"> </span><span class="n">allocate_c</span><span class="p">)</span>
<span class="w">          </span><span class="p">.</span><span class="n">precede</span><span class="p">(</span><span class="n">free</span><span class="p">);</span>
<span class="w">  </span>
<span class="w">  </span><span class="c1">// dump the graph without unfolding the cudaFlow</span>
<span class="w">  </span><span class="n">taskflow</span><span class="p">.</span><span class="n">dump</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="p">);</span>

<span class="w">  </span><span class="c1">// run the taskflow</span>
<span class="w">  </span><span class="n">executor</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">taskflow</span><span class="p">).</span><span class="n">wait</span><span class="p">();</span>

<span class="w">  </span><span class="c1">// dump the entire execution graph including unfolded cudaFlow</span>
<span class="w">  </span><span class="n">taskflow</span><span class="p">.</span><span class="n">dump</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">cout</span><span class="p">);</span>
<span class="p">}</span></pre><p>Within the cudaFlow, we create two host-to-device (H2D) tasks that copy data from <code>A</code> and <code>B</code> to <code>da</code> and <code>db</code>, one device-to-host (D2H) task that copies the result from <code>dc</code> to <code>C</code>, and one kernel task that launches <code>matmul</code> on the GPU (by default, GPU 0). H2D tasks precede the kernel and the kernel precedes the D2H task. These GPU operations form a GPU task graph managed by a cudaFlow. The first dump of the taskflow gives the following graph:</p><div class="m-graph"><svg style="width: 25.700rem; height: 18.800rem;" viewBox="0.00 0.00 257.30 188.00">
<g transform="scale(1 1) rotate(0) translate(4 184)">
<title>Taskflow</title>
<g class="m-node m-flat">
<title>p0x55d923794f10</title>
<ellipse cx="35.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="35.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_a</text>
</g>
<g class="m-node">
<title>p0x55d923795240</title>
<polygon points="153.65,-108 150.65,-112 129.65,-112 126.65,-108 95.65,-108 95.65,-72 153.65,-72 153.65,-108"/>
<text text-anchor="middle" x="124.65" y="-86.12" font-family="Helvetica,sans-Serif" font-size="10.00">cudaFlow</text>
</g>
<g class="m-edge">
<title>p0x55d923794f10&#45;&gt;p0x55d923795240</title>
<path d="M54.09,-146.5C65.51,-137.52 80.46,-125.76 93.66,-115.37"/>
<polygon points="95.57,-118.33 101.26,-109.39 91.24,-112.82 95.57,-118.33"/>
</g>
<g class="m-node m-flat">
<title>p0x55d923795350</title>
<ellipse cx="124.65" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="124.65" y="-14.12" font-family="Helvetica,sans-Serif" font-size="10.00">free</text>
</g>
<g class="m-edge">
<title>p0x55d923795240&#45;&gt;p0x55d923795350</title>
<path d="M124.65,-71.7C124.65,-64.41 124.65,-55.73 124.65,-47.54"/>
<polygon points="128.15,-47.62 124.65,-37.62 121.15,-47.62 128.15,-47.62"/>
</g>
<g class="m-node m-flat">
<title>p0x55d923795020</title>
<ellipse cx="124.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="124.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_b</text>
</g>
<g class="m-edge">
<title>p0x55d923795020&#45;&gt;p0x55d923795240</title>
<path d="M124.65,-143.7C124.65,-136.41 124.65,-127.73 124.65,-119.54"/>
<polygon points="128.15,-119.62 124.65,-109.62 121.15,-119.62 128.15,-119.62"/>
</g>
<g class="m-node m-flat">
<title>p0x55d923795130</title>
<ellipse cx="213.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="213.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_c</text>
</g>
<g class="m-edge">
<title>p0x55d923795130&#45;&gt;p0x55d923795240</title>
<path d="M195.21,-146.5C183.79,-137.52 168.84,-125.76 155.64,-115.37"/>
<polygon points="158.06,-112.82 148.04,-109.39 153.73,-118.33 158.06,-112.82"/>
</g>
</g>
</svg>
</div><p>A cudaFlow encapsulates a GPU task dependency graph similar to a <a href="classtf_1_1Subflow.html" class="m-doc">tf::<wbr />Subflow</a> (see <a href="SubflowTasking.html" class="m-doc">Subflow Tasking</a>). In order to visualize it, we need to execute the graph first and then dump the taskflow.</p><div class="m-graph"><svg style="width: 40.100rem; height: 36.700rem;" viewBox="0.00 0.00 401.30 367.25">
<g transform="scale(1 1) rotate(0) translate(4 363.25)">
<title>Taskflow</title>
<g class="m-cluster">
<title>cluster_p0x5558af971240</title>
<polygon points="79.65,-64 79.65,-351.25 224.65,-351.25 224.65,-64 79.65,-64"/>
<text text-anchor="middle" x="152.15" y="-337.75" font-family="Helvetica,sans-Serif" font-size="10.00">cudaFlow: cudaFlow</text>
</g>
<g class="m-node m-flat">
<title>p0x5558af970f10</title>
<ellipse cx="35.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="35.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_a</text>
</g>
<g class="m-node">
<title>p0x5558af971240</title>
<polygon points="216.65,-108 213.65,-112 192.65,-112 189.65,-108 158.65,-108 158.65,-72 216.65,-72 216.65,-108"/>
<text text-anchor="middle" x="187.65" y="-86.12" font-family="Helvetica,sans-Serif" font-size="10.00">cudaFlow</text>
</g>
<g class="m-edge">
<title>p0x5558af970f10&#45;&gt;p0x5558af971240</title>
<path d="M61.43,-149.13C85.14,-138.21 120.64,-121.86 147.96,-109.28"/>
<polygon points="149.36,-112.49 156.98,-105.12 146.43,-106.13 149.36,-112.49"/>
</g>
<g class="m-node m-flat">
<title>p0x5558af971350</title>
<ellipse cx="187.65" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="187.65" y="-14.12" font-family="Helvetica,sans-Serif" font-size="10.00">free</text>
</g>
<g class="m-edge">
<title>p0x5558af971240&#45;&gt;p0x5558af971350</title>
<path d="M187.65,-71.7C187.65,-64.41 187.65,-55.73 187.65,-47.54"/>
<polygon points="191.15,-47.62 187.65,-37.62 184.15,-47.62 191.15,-47.62"/>
</g>
<g class="m-node m-flat">
<title>p0x5558af971020</title>
<ellipse cx="268.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="268.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_b</text>
</g>
<g class="m-edge">
<title>p0x5558af971020&#45;&gt;p0x5558af971240</title>
<path d="M251.47,-146.15C241.26,-137.33 228.04,-125.91 216.29,-115.75"/>
<polygon points="218.66,-113.17 208.8,-109.28 214.08,-118.47 218.66,-113.17"/>
</g>
<g class="m-node m-flat">
<title>p0x5558af971130</title>
<ellipse cx="357.65" cy="-162" rx="35.65" ry="18"/>
<text text-anchor="middle" x="357.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">allocate_c</text>
</g>
<g class="m-edge">
<title>p0x5558af971130&#45;&gt;p0x5558af971240</title>
<path d="M330.71,-149.91C302.87,-138.44 259.11,-120.42 227.31,-107.33"/>
<polygon points="229.01,-104.24 218.43,-103.67 226.34,-110.72 229.01,-104.24"/>
</g>
<g class="m-node m-flat">
<title>p0x7f6fd8000b20</title>
<ellipse cx="188.65" cy="-306" rx="27.66" ry="18"/>
<text text-anchor="middle" x="188.65" y="-302.12" font-family="Helvetica,sans-Serif" font-size="10.00">H2D_a</text>
</g>
<g class="m-node">
<title>p0x7f6fd8000db0</title>
<polygon points="214.65,-252 164.65,-252 160.65,-248 160.65,-216 210.65,-216 214.65,-220 214.65,-252"/>
<polyline points="210.65,-248 160.65,-248"/>
<polyline points="210.65,-248 210.65,-216"/>
<polyline points="210.65,-248 214.65,-252"/>
<text text-anchor="middle" x="187.65" y="-230.12" font-family="Helvetica,sans-Serif" font-size="10.00" fill="white">matmul</text>
</g>
<g class="m-edge">
<title>p0x7f6fd8000b20&#45;&gt;p0x7f6fd8000db0</title>
<path d="M188.4,-287.7C188.3,-280.41 188.17,-271.73 188.06,-263.54"/>
<polygon points="191.56,-263.57 187.92,-253.62 184.56,-263.67 191.56,-263.57"/>
</g>
<g class="m-node m-flat">
<title>p0x7f6fd8000ce0</title>
<ellipse cx="187.65" cy="-162" rx="27.66" ry="18"/>
<text text-anchor="middle" x="187.65" y="-158.12" font-family="Helvetica,sans-Serif" font-size="10.00">D2H_c</text>
</g>
<g class="m-edge">
<title>p0x7f6fd8000db0&#45;&gt;p0x7f6fd8000ce0</title>
<path d="M187.65,-215.7C187.65,-208.41 187.65,-199.73 187.65,-191.54"/>
<polygon points="191.15,-191.62 187.65,-181.62 184.15,-191.62 191.15,-191.62"/>
</g>
<g class="m-node m-flat">
<title>p0x7f6fd8000c00</title>
<ellipse cx="115.65" cy="-306" rx="27.66" ry="18"/>
<text text-anchor="middle" x="115.65" y="-302.12" font-family="Helvetica,sans-Serif" font-size="10.00">H2D_b</text>
</g>
<g class="m-edge">
<title>p0x7f6fd8000c00&#45;&gt;p0x7f6fd8000db0</title>
<path d="M130.57,-290.5C139.54,-281.77 151.21,-270.43 161.65,-260.28"/>
<polygon points="163.88,-262.99 168.61,-253.51 159,-257.97 163.88,-262.99"/>
</g>
<g class="m-edge">
<title>p0x7f6fd8000ce0&#45;&gt;p0x5558af971240</title>
<path d="M187.65,-143.7C187.65,-136.41 187.65,-127.73 187.65,-119.54"/>
<polygon points="191.15,-119.62 187.65,-109.62 184.15,-119.62 191.15,-119.62"/>
</g>
</g>
</svg>
</div></section><section id="MatrixMultiplicationcudaFlowBenchmarking"><h2><a href="#MatrixMultiplicationcudaFlowBenchmarking">Benchmarking</a></h2><p>We run three versions of matrix multiplication, sequential CPU, parallel CPUs, and one GPU, on a machine of 12 Intel i7-8700 CPUs at 3.20 GHz and a Nvidia RTX 2080 GPU using various matrix sizes of <code>A</code>, <code>B</code>, and <code>C</code>.</p><table class="m-table"><thead><tr><th>A</th><th>B</th><th>C</th><th>CPU Sequential</th><th>CPU Parallel</th><th>GPU Parallel</th></tr></thead><tbody><tr><td>10x10</td><td>10x10</td><td>10x10</td><td>0.142 ms</td><td>0.414 ms</td><td>82 ms</td></tr><tr><td>100x100</td><td>100x100</td><td>100x100</td><td>1.641 ms</td><td>0.733 ms</td><td>83 ms</td></tr><tr><td>1000x1000</td><td>1000x1000</td><td>1000x1000</td><td>1532 ms</td><td>504 ms</td><td>85 ms</td></tr><tr><td>2000x2000</td><td>2000x2000</td><td>2000x2000</td><td>25688 ms</td><td>4387 ms</td><td>133 ms</td></tr><tr><td>3000x3000</td><td>3000x3000</td><td>3000x3000</td><td>104838 ms</td><td>16170 ms</td><td>214 ms</td></tr><tr><td>4000x4000</td><td>4000x4000</td><td>4000x4000</td><td>250133 ms</td><td>39646 ms</td><td>427 ms</td></tr></tbody></table><p>As the matrix size increases, the speed-up of GPU over CPUs becomes prominent. For example, at <code>4000x4000</code>, the GPU runtime is 585.8 times faster than the sequential CPU runtime and is 92.8 times faster than the parallel CPU solutions.</p></section>
      </div>
    </div>
  </div>
</article></main>
<div class="m-doc-search" id="search">
  <a href="#!" onclick="return hideSearch()"></a>
  <div class="m-container">
    <div class="m-row">
      <div class="m-col-m-8 m-push-m-2">
        <div class="m-doc-search-header m-text m-small">
          <div><span class="m-label m-default">Tab</span> / <span class="m-label m-default">T</span> to search, <span class="m-label m-default">Esc</span> to close</div>
          <div id="search-symbolcount">&hellip;</div>
        </div>
        <div class="m-doc-search-content">
          <form>
            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
          </form>
          <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
          <div id="search-help" class="m-text m-dim m-text-center">
            <p class="m-noindent">Search for symbols, directories, files, pages or
            modules. You can omit any prefix from the symbol or file path; adding a
            <code>:</code> or <code>/</code> suffix lists all members of given symbol or
            directory.</p>
            <p class="m-noindent">Use <span class="m-label m-dim">&darr;</span>
            / <span class="m-label m-dim">&uarr;</span> to navigate through the list,
            <span class="m-label m-dim">Enter</span> to go.
            <span class="m-label m-dim">Tab</span> autocompletes common prefix, you can
            copy a link to the result using <span class="m-label m-dim">⌘</span>
            <span class="m-label m-dim">L</span> while <span class="m-label m-dim">⌘</span>
            <span class="m-label m-dim">M</span> produces a Markdown link.</p>
          </div>
          <div id="search-notfound" class="m-text m-warning m-text-center">Sorry, nothing was found.</div>
          <ul id="search-results"></ul>
        </div>
      </div>
    </div>
  </div>
</div>
<script src="search-v2.js"></script>
<script src="searchdata-v2.js" async="async"></script>
<footer><nav>
  <div class="m-container">
    <div class="m-row">
      <div class="m-col-l-10 m-push-l-1">
        <p>Taskflow handbook is part of the <a href="https://taskflow.github.io">Taskflow project</a>, copyright © <a href="https://tsung-wei-huang.github.io/">Dr. Tsung-Wei Huang</a>, 2018&ndash;2024.<br />Generated by <a href="https://doxygen.org/">Doxygen</a> 1.9.6 and <a href="https://mcss.mosra.cz/">m.css</a>.</p>
      </div>
    </div>
  </div>
</nav></footer>
</body>
</html>
